Efficient state tracking for clusters

ABSTRACT

Exemplary method, system, and computer program product embodiments for efficient state tracking for clusters are provided. In one embodiment, by way of example only, in a distributed shared memory architecture, an asynchronous calculation of deltas and the views is performed while concurrently receiving client requests and concurrently tracking the client requests times. The results of the asynchronous calculation may be applied to each of the client requests that are competing for data of the same concurrency during a certain period with currently executing client requests. Additional system and computer program product embodiments are disclosed and provide related advantages.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of U.S. patent application Ser. No.13/351,132, filed on Jan. 16, 2012.

FIELD OF THE INVENTION

The present invention relates generally to computers, and moreparticularly, to efficient state tracking for clusters in a computingenvironment.

DESCRIPTION OF THE RELATED ART

In today's society, computer systems are commonplace. Computer systemsmay be found in the workplace, at home, or at school. Computer systemsmay include data storage systems, or disk storage systems, to processand store data. A storage system may include various storage components,such as one or more disk drives configured in a storage environment. Forexample, the storage environment may include a number of disk drivesimplemented in an array, such as a Redundant Array of Independent Disks(RAID) topology, to provide data security in the event of a hardware orsoftware failure. The storage environment may also include other storagecomponents, such as controllers and interfaces to mange the flow ofdata. Moreover, the computer system may include a complex dataprocessing system or computing environment. A data processing systemoften requires computational resources availability requirements thatcannot be achieved by a single computer.

SUMMARY OF THE DESCRIBED EMBODIMENTS

As previously mentioned, a computer system may include a complex dataprocessing system or computing environment. The data processing systemoften requires computational resources or availability requirements thatcannot be achieved by a single computer. In such cases, a number ofcomputers and storage systems may be architecturally arranged to form acluster for networking several data processing systems together for thepurpose of providing continuous resource availability and for sharingworkload. Within the cluster deployment, for example, a cluster may beone node or, more commonly, a set of multiple nodes coordinating accessto a set of shared storage subsystems. In a clustered environment,states may be replicated across the nodes of the cluster in form ofdistributed shared memory (DSM) providing efficient access to the sharedstate for all applications on every node. The way in which externallyhosted applications (“clients”, e.g., a management application or userfront end) track a cluster's state presents a serious challenge.

Accordingly, and in view of the foregoing, various method, system, andcomputer program product embodiments for efficient state tracking forclusters are provided. In one embodiment, by way of example only, in adistributed shared memory architecture, an asynchronous calculation ofdeltas and the views is performed while concurrently receiving clientrequests and concurrently tracking the client requests times. Theresults of the asynchronous calculation may be applied to each of theclient requests that are competing for data of the same concurrencyduring a certain period with currently executing client requests.Additional system and computer program product embodiments are disclosedand provide related advantages.

The foregoing summary has been provided to introduce a selection ofconcepts in a simplified form that are further described below in theDetailed Description. This Summary is not intended to identify keyfeatures or essential features of the claimed subject matter, nor is itintended to be used as an aid in determining the scope of the claimedsubject matter. The claimed subject matter is not limited toimplementations that solve any or all disadvantages noted in thebackground.

BRIEF DESCRIPTION OF THE DRAWINGS

In order that the advantages of the invention will be readilyunderstood, a more particular description of the invention brieflydescribed above will be rendered by reference to specific embodimentsthat are illustrated in the appended drawings. Understanding that thesedrawings depict embodiments of the invention and are not therefore to beconsidered to be limiting of its scope, the invention will be describedand explained with additional specificity and detail through the use ofthe accompanying drawings, in which:

FIG. 1 illustrates a computer environment having an example storagedevice in which aspects of the present invention may be realized;

FIG. 2 illustrates an exemplary block diagram showing a hardwarestructure of a data storage system in a computer system in which aspectsof the present invention may be realized;

FIG. 3 is a flowchart illustrating an exemplary method for efficientstate tracking for clusters; and

FIG. 4 is an exemplary time-line diagram illustrating the asynchronousconstituent processes of the method, with time advancing from left towrite.

DETAILED DESCRIPTION OF CERTAIN EMBODIMENTS

A distributed shared memory (DSM) presents an area of shared memory fortwo or more servers in the cluster. DSM may be superior even whencompared to pure-in-memory database as it exposes all data into thevirtual address space of a process. Thus, the need to query data in aparticular language (e.g., SQL) through a database service iseliminated. Within a clustered appliance, states may be replicatedacross the nodes of the cluster in form of the DSM providing efficientaccess to the shared state for all applications on every node. The datain this memory may be structured as a tree of arbitrary complexstructure, for example, or with rows of tables (similar to relationaldatabases). Transactional services may ensure that the DSM is at alltimes consistent across all nodes. However, as mentioned previously, themanner and method in which externally hosted applications (“clients”,e.g., a management application or user front end) track a cluster'sstate presents a serious challenge. It should be noted that thedefinitions described herein are applicable to the illustratedembodiments by way of example, and other definitions, as commonlyunderstood in the art, may be applicable as well for describing thefunctionality and purpose of the illustrated embodiments of the presentinvention.

In one embodiment, by way of example only, the mechanisms of theillustrated embodiments serve to distinguish between internally visibledata (“state”) and externally visible data (“views”). The internallyvisible data (called “state”) is the ensemble of variables needed torepresent the internal state of every application in the cluster. Theinternally visible data is stored in the DSM and governed by itstransactional services. Sharing the state across the cluster allowsapplications to coordinate their actions and enables seamless failoverin the face of hardware and/or software failures. Conversely, theexternally visible data (called “views”) is a representation of thestate tailored for management and monitoring applications, and thus,ultimately for the human user. The views are calculated from the stateby filtering, restructuring, aggregation, or simple transformations,such as translation into different units. The views are stored in thememory of the application providing the views, and specifically, not inthe DSM.

Where these views are very voluminous, the clients may require a way totrack them by querying differential updates instead of full updates.Applying such differential updates (“deltas”) allows clients to maintaina local replica of the appliance's views. The deltas may be thedifference between two views. Such deltas are kept in a circular bufferon the appliance. If a client requests a delta across a transactionrange not covered by the buffer (because of its limited size), a fullview resynchronization is needed.

In one embodiment, by way of example only, the implementation of such ascheme may require that views and deltas are recalculated after everytransaction by the node initiating the transaction and distributed aspart of the DSM. By this scheme, the views and deltas may be triviallyconsistent across the cluster and ensured by the same transactionalservices used for the state itself (i.e., views and deltas are part ofthe state). However, the calculation of the views may be complex andtake significant time. Moreover, as the views perform aggregations,updating the views has a near constant cost, which is independent oftransaction size.

In situations where fast transactions are a necessity for an efficientfunctioning of the appliance, calculating the views and deltas in acritical path (e.g., synchronous calculation) may present variousproblems. Specifically, the synchronous calculation may dominate thecost of a transaction itself and thus severely limit the effectivenessof the appliance.

Thus, in a distributed shared memory architecture, the mechanisms of thepresent invention provide for an asynchronous calculation of the views,as well as the deltas (the difference between the newly calculated viewsand the previously calculated views). While at all times only one copyof the views (the most recent one) is kept in memory, the deltas arekept in a circular buffer large enough to keep deltas for manyrecalculations of the views. This allows clients to forward theirreplica of the views using deltas only, provided they query such deltasbefore the circular buffer is exhausted.

Client requests are received asynchronously to the calculations. When arequest is received, the arrival time, or equivalently, the sequencenumber of the most recent state transaction, is recorded. Unansweredrequests are kept in a queue. Requests are answered whenever a view ofsufficient currency becomes available, which may be either immediately(the current view is already based on the most recent transaction); oronce the current calculation finishes (the current calculationincorporates this transaction); or once the next calculation (yet to bestarted) finishes (the current view is out of date, and an ongoingcalculation, if any, is based on an older transaction). The upper boundfor the response latency for any request is thus intrinsically the timerequired for two view calculations (ignoring network delays). While norequests are pending, view calculations are not performed. The use ofthe term “currency” rather than “concurrency” is intentional. In oneembodiment, use of the term “currency” is intended to refer to (amongother things) the degree of how current something is, while“concurrency” is about two things co-occurring.

In one embodiment, the mechanisms take the asynchronous calculation ofviews and deltas out of the critical transaction path, performing themasynchronously instead of synchronously. Since this allows transactionsto be performed at a much faster rate, the implication is that views anddeltas cannot be calculated after every transaction. Moreover, thecalculations of views and deltas may be performed only on demand therebyimplicitly folding deltas to the maximum degree.

Turning to FIG. 1, an example computer system 10 is depicted in whichaspects of the present invention may be realized. Computer system 10includes central processing unit (CPU) 12, which is connected to massstorage device(s) 14 and memory device 16. Mass storage devices mayinclude hard disk drive (HDD) devices, which may be configured in aredundant array of independent disks (RAID). Memory device 16 mayinclude such memory as electrically erasable programmable read onlymemory (EEPROM) or a host of related devices. Memory device 16 and massstorage device 14 are connected to CPU 12 via a signal-bearing medium.In addition, CPU 12 is connected through communication port 18 to acommunication network 20, having an attached plurality of additionalcomputer systems 22 and 24. The computer system 10 may include one ormore processor devices (e.g., CPU 12) and additional memory devices 16for each individual component of the computer system 10.

FIG. 2 is an exemplary block diagram 200 showing a hardware structure ofa data storage system in a computer system according to the presentinvention. Host computers 210, 220, 225, are shown, each acting as acentral processing unit for performing data processing as part of a datastorage system 200. The cluster hosts/nodes (physical or virtualdevices), 210, 220, and 225 may be one or more new physical devices orlogical devices to accomplish the purposes of the present invention inthe data storage system 200. In one embodiment, by way of example only,a data storage system 200 may be implemented as IBM® System Storage™DS8000™. A Network connection 260 may be a fibre channel fabric, a fibrechannel point to point link, a fibre channel over ethernet fabric orpoint to point link, a FICON or ESCON I/O interface, any other I/Ointerface type, a wireless network, a wired network, a LAN, a WAN,heterogeneous, homogeneous, public (i.e. the Internet), private, or anycombination thereof. The hosts, 210, 220, and 225 may be local ordistributed among one or more locations and may be equipped with anytype of fabric (or fabric channel) (not shown in FIG. 2) or networkadapter 260 to the storage controller 240, such as Fibre channel, FICON,ESCON, Ethernet, fiber optic, wireless, or coaxial adapters. Datastorage system 200 is accordingly equipped with a suitable fabric (notshown in FIG. 2) or network adapter 260 to communicate. Data storagesystem 200 is depicted in FIG. 2 comprising storage controllers 240 andcluster hosts 210, 220, and 225. The cluster hosts 210, 220, and 225 mayinclude cluster nodes.

To facilitate a clearer understanding of the methods described herein,storage controller 240 is shown in FIG. 2 as a single processing unit,including a microprocessor 242, system memory 243 and nonvolatilestorage (“NVS”) 216, which will be described in more detail below. It isnoted that in some embodiments, storage controller 240 is comprised ofmultiple processing units, each with their own processor complex andsystem memory, and interconnected by a dedicated network within datastorage system 200. Storage 230 may be comprised of one or more storagedevices, such as storage arrays, which are connected to storagecontroller 240 (by a storage network) with one or more cluster hosts210, 220, and 225 connected to each storage controller 240.

In some embodiments, the devices included in storage 230 may beconnected in a loop architecture. Storage controller 240 manages storage230 and facilitates the processing of write and read requests intendedfor storage 230. The system memory 243 of storage controller 240 storesprogram instructions and data, which the processor 242 may access forexecuting functions and method steps of the present invention forexecuting and managing storage 230 as described herein. In oneembodiment, system memory 243 includes, is in association with, or is incommunication with the operation software 250 for performing methods andoperations described herein. As shown in FIG. 2, system memory 243 mayalso include or be in communication with a cache 245 for storage 230,also referred to herein as a “cache memory”, for buffering “write data”and “read data”, which respectively refer to write/read requests andtheir associated data. In one embodiment, cache 245 is allocated in adevice external to system memory 243, yet remains accessible bymicroprocessor 242 and may serve to provide additional security againstdata loss, in addition to carrying out the operations as described inherein.

In some embodiments, cache 245 is implemented with a volatile memory andnon-volatile memory and coupled to microprocessor 242 via a local bus(not shown in FIG. 2) for enhanced performance of data storage system200. The NVS 216 included in data storage controller is accessible bymicroprocessor 242 and serves to provide additional support foroperations and execution of the present invention as described in otherfigures. The NVS 216, may also referred to as a “persistent” cache, or“cache memory” and is implemented with nonvolatile memory that may ormay not utilize external power to retain data stored therein. The NVSmay be stored in and with the cache 245 for any purposes suited toaccomplish the objectives of the present invention. In some embodiments,a backup power source (not shown in FIG. 2), such as a battery, suppliesNVS 216 with sufficient power to retain the data stored therein in caseof power loss to data storage system 200. In certain embodiments, thecapacity of NVS 216 is less than or equal to the total capacity of cache245.

Storage 230 may be physically comprised of one or more storage devices,such as storage arrays. A storage array is a logical grouping ofindividual storage devices, such as a hard disk. In certain embodiments,storage 230 is comprised of a JBOD (Just a Bunch of Disks) array or aRAID (Redundant Array of Independent Disks) array. A collection ofphysical storage arrays may be further combined to form a rank, whichdissociates the physical storage from the logical configuration. Thestorage space in a rank may be allocated into logical volumes, whichdefine the storage location specified in a write/read request.

In one embodiment, by way of example only, the storage system as shownin FIG. 2 may include a logical volume, or simply “volume,” may havedifferent kinds of allocations. Storage 230 a, 230 b and 230 n are shownas ranks in data storage system 200, and are referred to herein as rank230 a, 230 b and 230 n. Ranks may be local to data storage system 200,or may be located at a physically remote location. In other words, alocal storage controller may connect with a remote storage controllerand manage storage at the remote location. Rank 230 a is shownconfigured with two entire volumes, 234 and 236, as well as one partialvolume 232 a. Rank 230 b is shown with another partial volume 232 b.Thus volume 232 is allocated across ranks 230 a and 230 b. Rank 230 n isshown as being fully allocated to volume 238—that is, rank 230 n refersto the entire physical storage for volume 238. From the above examples,it will be appreciated that a rank may be configured to include one ormore partial and/or entire volumes. Volumes and ranks may further bedivided into so-called “tracks,” which represent a fixed block ofstorage. A track is therefore associated with a given volume and may begiven a given rank.

The storage controller 240 may include an asynchronous calculationmodule 255 and state-tracking module 257. The asynchronous calculationmodule 255 and state-tracking module 257 may work in conjunction witheach and every component of the storage controller 240, the hosts 210,220, 225, and storage devices 230. Both the asynchronous calculationmodule 255 and state-tracking module 257 may be structurally onecomplete module or may be associated and/or included with otherindividual modules. The asynchronous calculation module 255 andstate-tracking module 257 may also be located in the cache 245 or othercomponents.

The storage controller 240 includes a control switch 241 for controllingthe fiber channel protocol to the host computers 210, 220, 225, amicroprocessor 242 for controlling all the storage controller 240, anonvolatile control memory 243 for storing a microprogram (operationsoftware) 250 for controlling the operation of storage controller 240,data for control and each table described later, cache 245 fortemporarily storing (buffering) data, and buffers 244 for assisting thecache 245 to read and write data, a control switch 241 for controlling aprotocol to control data transfer to or from the storage devices 230,and asynchronous calculation module 255 and state-tracking module 257 inwhich information may be set. Multiple buffers 244 may be implementedwith the present invention to assist with the operations as describedherein. In one embodiment, the cluster hosts/nodes, 210, 220, 225 andthe storage controller 240 are connected through a network adaptor (thiscould be a fibre channel) 260 as an interface i.e., via a switch called“fabric.”

In one embodiment, the host computers or one or more physical or virtualdevices, 210, 220, 225 and the storage controller 240 are connectedthrough a network adaptor (this could be a fibre channel) 260 as aninterface i.e., via a switch called “fabric.” In one embodiment, theoperation of the system shown in FIG. 2 will be described. Themicroprocessor 242 may control the memory 243 to store commandinformation from the host device (physical or virtual) 210 andinformation for identifying the host device (physical or virtual) 210.The control switch 241, the buffers 244, the cache 245, the operatingsoftware 250, the microprocessor 242, memory 243, NVS 216, asynchronouscalculation module 255 and state tracking module 257 are incommunication with each other and may be separate or one individualcomponent(s). Also, several, if not all of the components, such as theoperation software 250 may be included with the memory 243. Each of thecomponents within the devices shown may be linked together and may be incommunication with each other for purposes suited to the presentinvention.

In one embodiment, the views are updated. An asynchronous calculation ofdeltas and views is performed. Concurrently, client requests arereceived and their times of arrival are tracked. The results of theasynchronous calculation may be applied to each of the client requeststhat are competing for data of the same concurrency during a certainperiod with currently executing client requests. This embodiment isoutlined in the FIG. 3, as described below.

FIG. 3 is a flowchart illustrating an exemplary method 300 for efficientstate tracking for clusters. The method 300 begins (step 302). Anasynchronous calculation of deltas and the views is performed (step304). The views may be calculated from the state by filtering,restructuring, aggregation, or simple transformations, such astranslation into different units. The views are stored in the memory ofthe application providing the views, and specifically, not in the DSM.Concurrently and/or in parallel, client requests are received and theclient requests times are tracked (step 306). It should be noted thatthe tracking of the client request may be performed in parallel with theother operations and steps described herein. The results of theasynchronous calculation may be applied to each of the client requeststhat are competing for data of the same concurrency during a certainperiod with currently executing client requests (step 308). The method300 ends (step 310).

With an asynchronous calculation of views and deltas, the transactionrate may be too fast for the calculation of views and deltas to finishbefore the next transaction arrives. Recording the state after eachtransaction (to buffer calculations) may not be a possibility, as itrequires a theoretically unbounded amount of memory. For example,consider the following four transactions:

1. add row A with value 12. change row A to value 2 and add row B with value 1 (value 3)3. remove row A and change row B to value 44. remove row B

Assume that the view calculation is trivial in this case (i.e., theidentity function). Now assume that in node X of a cluster the timingsare such that the deltas are calculated after transactions 2 and 4. Innode Y, deltas are calculated only after transaction 4. Thus, the deltasmay be folded. The following reflect the “folded deltas” that thesenodes compute:

Node X:

1-2. add to row A (value 2), add to row B (value 3)

3-4. remove row A, remove row B

Node Y:

1-4.—

Specifically, in Node Y, the folding was such that the resulting deltawas empty. It is important to realize that deltas are not calculatedfrom “elementary deltas” (i.e., a transaction log). If they were, itwould simply imply a theoretically unbounded memory region to store thelog of elementary deltas. Rather, the deltas are calculated by directcomparison between snapshots of state and views, a “most recent”snapshot (from the time when the calculation was started), and a“previous” snapshot (the most recent snapshot from previouscalculation). Thus, the space requirement is precisely two full copiesof DSM and views.

In a general sense, folding may be described as the process of combiningseveral alterations of the same data into a simpler resulting (net)difference. Folding is not achieved by actually combining suchalterations, but is incurred implicitly by comparing the new state tothe old state. This state comparison will yield the net difference,rather than the potentially many alterations that took place in between.

In one embodiment, the folding provides a valuable tool for viewtracking as it reduces the amount of data that needs to be transmittedto the clients. Also, the folding allows for a reduction in theexhaustion of circular buffers of deltas. However, because of the lackof coordination across nodes, delta sets vary from node to node.Specifically in the above example, it would be impossible to query nodeY for the “delta since transaction 2”, as this transaction was foldedinto a single indivisible delta for transactions 1-4. The same clientsquerying the same node is a necessity. However, this is unproblematic interms of load balancing, as load balancing is about balancing manyclients, rather than balancing many requests from one client (who haslimited bandwidth).

Thus, in one embodiment, the calculations of views and deltas may beperformed as soon as a calculation is complete and then the nextcalculation may commence (for whichever transaction is the most recent).In so doing, the mechanisms of the present invention preform anon-demand calculation. In other words, the asynchronous calculation maybe performed only once a request for “deltas since transaction X” isreceived. This on-demand calculation leads to the maximum amount offolding achievable.

The mechanisms of the present invention also seek to address and improveefficiency upon concurrent requests by multiple clients on the samenode. For example, on a particular node, consider that at time t1, arequest from client C1 is received. Immediately, a view and deltacalculation may be started (for current transaction T1). Shortlyafterwards, at time t2, a request from client C2 is received. At time t3(still during the calculation) a new transaction T2 arrives at the DSM.At time t4 the calculation of view and delta for T1 finished, and C1 isserved. Now the crucial observation is that C2's request was received ata time where T1 was still the “current” transaction. Thus, it ispermissible to also serve C2 with delta and view for T1, rather thancalculating view and delta for T2.

In one embodiment, the mechanisms do not record the “current transactionat request time” for each request. In the event of a flurry of requestsand assuming a steady flow of transactions at a higher rate thancompletion of calculations for the views and deltas, the mechanism seekto avoid calculating views and deltas for every single request, leadingto requests being delayed by the number of queued requests times thetime for the view and delta calculations. Conversely, the latency for arequest may be bounded by the time needed for exactly two views anddelta calculations (in the worst case where t1<t3<t2 in very narrowsuccession). Such concepts are further explained below in FIG. 4.

FIG. 4 is an exemplary time-line diagram illustrating the asynchronousconstituent processes of the method, with time advancing from left toright. In one embodiment, by way of example only, three asynchronousprocesses (represented as the large arrows), with time advancing fromleft to right, are depicted. The first asynchronous process is thestream of transactions, as perceived in the present node. The secondasynchronous process is the “update-serve” (US) thread, which alternatesbetween calculating views & deltas and serving queued queries. The thirdasynchronous process is the thread that receives queries, records thequeries run times (or current transaction) at the time of receipt, andqueues them.

Initially the US thread is idle, as the request queue is empty (nopending request). Now the mechanisms of the illustrated embodimentsreceive R1; this triggers the US thread to calculate views & deltasbased on the most recent transaction. (emphasis added). The mechanismswill first, atomically take snapshots of the current state (by whichevertechnique is applied). The views and deltas will be calculated on thesnapshot, as the DSM may move on during the calculation, which it doesso in this example. Next, the US thread performs the lengthy calculationof views and deltas. Meanwhile the mechanisms have the following events:R2, T2, R3, T3 (in this order). After the calculation on the US threadis finished, the thread may serve R1 and R2. R2 may be served from thesame image (views and deltas), as it arrived before T2.

On the other hand, R3 may not be served based on the same image, as itarrived after T2. Next, after serving R1 and R2, the US thread will onceagain atomically take a snapshot of the state (discarding the previoussnapshot). However, it should be noted the mechanisms atomically takesnapshots of the state based on the most recent transaction T3, ratherthan T2—even though R3 came in before T3. (emphasis added). Theadvantage is obvious. In this way, additional requests (that are yet tocome) may be served by the same image T3 (and R4, R5 indeed will). Onthe other hand, if the mechanisms could snapshot T2 (which it could notdo since it is too late), then any future requests would surely requireyet another calculation, as T3 has already arrived.

The more interesting question is why is it correct to serve the T3 stateto a request that arrived before T3? This is in the nature ofasynchronous systems: if a client makes a request that (on the userside) begins at t begin and returns at tend, then it cannot have anyexpectation as to when exactly anything was executed on the serverside—it could be that the network delays were such that most of the timeelapsed before the server even saw the request, and everything happenedright at t_end; or vice versa. The entire time is essentially a blackoutperiod for the client, during which, at some point, the request wasexecuted. Thus, in this case, where R3 comes in while T2 was current,and returns when T5 is current, any transaction level between T2 and T5would be a valid answer.

Returning to the example and FIG. 4, during the calculation for thestate T3, the mechanisms have the following events: R4, R5, T4, R6, T5,R7 (in this order). Since both R4 and R5 arrive before T4, these can beserved along with R3 (based on transaction T3). R6 and R7 however needto be served after yet another calculation (not illustrated in FIG. 4).This calculation would be based on T5 (again the most recenttransaction).

As will be appreciated by one skilled in the art, aspects of the presentinvention may be embodied as a system, method or computer programproduct. Accordingly, aspects of the present invention may take the formof an entirely hardware embodiment, an entirely software embodiment(including firmware, resident software, micro-code, etc.) or anembodiment combining software and hardware aspects that may allgenerally be referred to herein as a “circuit,” “module” or “system.”Furthermore, aspects of the present invention may take the form of acomputer program product embodied in one or more computer readablemedium(s) having computer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that may contain, or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wired, optical fiber cable, RF, etc., or any suitable combination of theforegoing. Computer program code for carrying out operations for aspectsof the present invention may be written in any combination of one ormore programming languages, including an object oriented programminglanguage such as Java, Smalltalk, C++ or the like and conventionalprocedural programming languages, such as the “C” programming languageor similar programming languages. The program code may execute entirelyon the user's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Aspects of the present invention are described above with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, may be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that may direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks. The computer program instructions may also beloaded onto a computer, other programmable data processing apparatus, orother devices to cause a series of operational steps to be performed onthe computer, other programmable apparatus or other devices to produce acomputer implemented process such that the instructions which execute onthe computer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

The flowchart and block diagram in the above figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock might occur out of the order noted in the figures. For example,two blocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, may be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

While one or more embodiments of the present invention have beenillustrated in detail, one of ordinary skill in the art will appreciatethat modifications and adaptations to those embodiments may be madewithout departing from the scope of the present invention as set forthin the following claims.

What is claimed is:
 1. A method for efficient state tracking forclusters by a processor device in a distributed shared memoryarchitecture, the method comprising: performing an asynchronouscalculation of deltas and the views while concurrently receiving clientrequests and concurrently tracking client requests times; and applyingresults of the asynchronous calculation to each of the client requeststhat are competing for data of the same concurrency during a certainperiod with currently executing client requests.
 2. The method of claim1, further including, in conjunction with the applying, executing eachof the competing client requests occurring after the certain period onthe updated views, wherein all of the client requests are updated. 3.The method of claim 1, further including, in conjunction with theperforming, folding the deltas to a maximum degree.
 4. The method ofclaim 1, further including, performing the asynchronous calculation forthe deltas by a direct comparison between one of a plurality of snapshotstates and the views, wherein the plurality of snapshots states includeat least one of a most recent snapshot and a previous snapshot.
 5. Themethod of claim 4, wherein the most recent snapshot is a state of theasynchronous calculation when initially commenced and the previoussnapshots is the state of the most recent snapshot and a last performedasynchronous calculation.
 6. The method of claim 1, further includingperforming the asynchronous calculation on demand.
 7. The method ofclaim 1, further including performing the asynchronous calculation onlyupon receipt of the client requests for the deltas calculated since aspecified client request.
 8. The method of claim 1, further includingbounding the latency for the client requests by a time needed for theasynchronous calculation of two of the views and the deltas.